This article showcases a step-by-step guide for creating an extensible, and production-ready approach for CI/CD pipelines. Utilizing a Go monorepo, GitLab CI, and Go templating has several key advantages, including seamless integration with GitLab, a flexible template engine, and declarative pipeline logic.

1. Introduction

In modern software development, the adoption of monorepos has become increasingly common, especially in organizations practicing microservices architecture. A monorepo simplifies dependency management, enforces consistency across services, and improves developer velocity. However, the operational complexity of setting up robust CI/CD pipelines for each service within the monorepo can be daunting.

This article presents an advanced CI/CD architecture built with Go and GitLab CI, leveraging the power of Go templates for dynamic pipeline generation. It examines two codebases:

• monorepo-go-main: A monorepo housing multiple Go services.
• ci-pipeline-main: A utility written in Go that reads configuration and uses templates to generate .gitlab-ci.yml files dynamically.

The approach combines the flexibility of Go with the power of GitLab CI to deliver a scalable, maintainable, and DRY CI/CD pipeline, making it a strong candidate for large teams and enterprise-grade systems.

2. Background and Motivation

Monorepos consolidate multiple services into a single repository, which makes code sharing, refactoring, and dependency upgrades much easier. However, the CI/CD implications of this approach are non-trivial. Traditionally, every service might come with its own .gitlab-ci.yml or Jenkinsfile, leading to excessive duplication, inconsistencies, and increased maintenance overhead.

Moreover, teams often face challenges like:

• Keeping pipeline definitions in sync across services
• Managing build dependencies across a shared module
• Avoiding unnecessary rebuilds when only one service changes
• Onboarding new services without duplicating configuration

The motivation behind this setup is to address these challenges by introducing a highly modular, configuration-driven pipeline generation mechanism.

STAY TUNED

Learn more about DevOpsCon

Leave this field empty if you're human:

3. Architecture Overview

The entire system revolves around a configuration-driven template engine that generates a unified CI/CD pipeline per commit. Each microservice in the monorepo is treated as an autonomous unit that can be built, tested, and deployed independently within the GitLab ecosystem.

Architectural Goals

• Declarative Configuration: Services are defined in pipeline_config.yaml.
• Templated Pipelines: A Go template is used to render the CI file.
• Service Autonomy: Each service has its own isolated testing block.
• Code Generation: The CI pipeline is generated programmatically.
• Dockerization: A Dockerfile encapsulates the CI generation process for portability.

This architecture ensures that onboarding a new service is a one-line change in the config file, without needing to manually edit YAML logic.

4. Deep Dive: monorepo-go-main

Folder Structure

monorepo-go-main/
├── .gitlab-ci.yml
├── go.mod
├── pipeline_config.yaml
├── README.md
├── service-1/
│   ├── main.go
│   └── addition_test.go
├── service-2/
│   ├── main.go
│   └── subtraction_test.go
└── service-3/
    ├── main.go
    └── multiplication_test.go

Module Definition (go.mod)

module monorepo-go-main

go 1.20

This ensures all services share the same Go toolchain and dependency tree, promoting consistency in build behavior across the repo.

pipeline_config.yaml

services:
  - name: service-1
  - name: service-2
  - name: service-3

This configuration file becomes the single source of truth. You can imagine a scenario where this config file is automatically generated or updated by a higher-level orchestration tool or service discovery mechanism.

Example Service: service-1

package main

import "fmt"

func add(a, b int) int {
	return a + b
}

func main() {
	fmt.Println("Addition Result:", add(5, 3))
}

Each service is a minimal but functional Go program with unit tests. This structure makes testing and deployment granular and independent.

5. Dynamic Pipeline Generator: ci-pipeline-main

Folder Layout

ci-pipeline-main/
├── .gitlab-ci.yml (optional example)
├── Dockerfile
├── gitlab-ci.tmpl
└── gitlab.go

This utility acts as a CLI tool to be run locally or inside CI, generating the full.gitlab-ci.ymlfile based on service declarations.

Understanding gitlab.go

package main

import (
	"os"
	"text/template"
	"gopkg.in/yaml.v2"
	"io/ioutil"
)

type Config struct {
	Services []struct {
		Name string `yaml:"name"`
	} `yaml:"services"`
}

func main() {
	data, _ := ioutil.ReadFile("../monorepo-go-main/pipeline_config.yaml")
	var config Config
	yaml.Unmarshal(data, &config)
	tmpl, _ := template.ParseFiles("gitlab-ci.tmpl")
	tmpl.Execute(os.Stdout, config)
}

This program does three things:

1. Reads the YAML config for services
2. Parses the template file
3. Outputs the populated CI pipeline

### Why Go Templates?

Go’s text/template package provides a safe, readable, and powerful tool for creating infrastructure as code. It supports loops, conditions, and functions. Unlike Jinja2 or other alternatives, it integrates naturally into Go-based workflows.

Kubernetes Training (German only)

Entdecke die Kubernetes Trainings für Einsteiger und Fortgeschrittene mit DevOps-Profi Erkan Yanar

Mehr erfahren

6. Template Language in Action

Template: gitlab-ci.tmpl

stages:
  - test

{{ range .Services }}
{{ .Name }}:
  stage: test
  script:
    - cd {{ .Name }}
    - go test ./...
{{ end }}

This loops over all services and creates individual test jobs.

Example Output

stages:
  - test

service-1:
  stage: test
  script:
    - cd service-1
    - go test ./...

service-2:
  stage: test
  script:
    - cd service-2
    - go test ./...

This concise pattern removes redundancy and encourages consistency.

7. Dockerfile: Portable Generator

Dockerfile

FROM golang:1.20

WORKDIR /app
COPY . .
RUN go build -o generator gitlab.go
ENTRYPOINT ["./generator"]

The Dockerfile ensures that the generator can run anywhere, which is crucial for reproducible builds in CI environments.

8. Workflow Summary

End-to-End Usage

1. Clone both repositories.
2. Add new services to monorepo-go-main/service-X.
3. Update pipeline_config.yaml with the new service.
4. Run the Go generator locally or in your CI system.
5. GitLab executes the generated .gitlab-ci.yml.

This allows pipelines to evolve alongside the code without introducing errors or regressions.

Advantages

• 🚀 No YAML duplication.
• ⚙️ Configuration is code.
• 🔒 Easier to audit and validate.
• 🧪 Unit testable pipeline logic.

9. Scaling the Pattern

Real-World Scenario

Imagine scaling from 3 to 50 microservices. Traditional approaches would involve maintaining 50.gitlab-ci.ymlblocks — one per service. With this generator:

• The pipeline_config.yaml grows by 1 line per service.
• No extra CI logic is added manually.
• The Go code and template remain unchanged.

Adding New Capabilities

You can easily extend the template:

• Add linting stages (golangci-lint)
• Build docker images per service
• Deploy to separate Kubernetes namespaces
• Cache modules or test results

10. Observability & Auditing

Logging Pipeline Execution

You can enhance the script with logging outputs to file or console.

Slack Notifications

Add notification blocks to the template:

  after_script:
    - curl -X POST --data "job $CI_JOB_NAME finished" $SLACK_HOOK

Static Analysis

You can write a linter that validates the final .gitlab-ci.yml for syntax and best practices.

11. Testing the Template Engine

Unit Testing in Go

Write test cases for the generator:

func TestPipelineGeneration(t *testing.T) {
	config := Config{
		Services: []struct{ Name string }{{"svc1"}, {"svc2"}},
	}
	tmpl, _ := template.ParseFiles("gitlab-ci.tmpl")
	var buf bytes.Buffer
	tmpl.Execute(&buf, config)

	if !strings.Contains(buf.String(), "svc1") {
		t.Fatal("Expected svc1 in output")
	}
}

This ensures the integrity of CI logic as templates evolve.

12. Conclusion

This article showcases a robust, extensible, and production-ready approach to managing CI/CD pipelines in a Go monorepo using GitLab CI and Go templating. It combines configuration as code with portable, repeatable pipeline generation.

The key advantages:

• Declarative and DRY pipeline logic
• Portable via Docker
• Flexible and extensible template engine
• Seamlessly integrates with GitLab

Whether you’re in a startup or an enterprise, this design can dramatically simplify your DevOps lifecycle.